Trading Off Perception with Internal State: Reinforcement Learning and Analysis of Q-Elman Networks in a Markovian Task
نویسندگان
چکیده
A Markovian reinforcement learning task can be dealt with by learning a direct mapping from states to actions or values, or from state-action pairs to values. However, this may involve a difficult pattern recognition problem when the state space is large. This paper shows that using internal state, called “supportive state”, may alleviate this problem—presenting an argument against the tendency to almost automatically use a direct mapping when the task is Markovian. This point is demonstrated in simulation experiments of an agent controlled by a neural network capable of learning the strategy of direct mapping as well as internal state, combining Q( )-learning and recurrent neural networks in a new way. The trade-off between the two strategies is investigated in more detail, focusing in particular on border cases.
منابع مشابه
Totally Model-Free Reinforcement Learning by Actor-Critic Elman Networks in Non-Markovian Domains
In this paper we describe how an actor critic rein forcement learning agent in a non Markovian domain nds an optimal sequence of actions in a totally model free fashion that is the agent neither learns transitional probabilities and associated rewards nor by how much the state space should be augmented so that the Markov prop erty holds In particular we employ an Elman type re current neural ne...
متن کاملMulticast Routing in Wireless Sensor Networks: A Distributed Reinforcement Learning Approach
Wireless Sensor Networks (WSNs) are consist of independent distributed sensors with storing, processing, sensing and communication capabilities to monitor physical or environmental conditions. There are number of challenges in WSNs because of limitation of battery power, communications, computation and storage space. In the recent years, computational intelligence approaches such as evolutionar...
متن کاملReinforcement Learning in Neural Networks: A Survey
In recent years, researches on reinforcement learning (RL) have focused on bridging the gap between adaptive optimal control and bio-inspired learning techniques. Neural network reinforcement learning (NNRL) is among the most popular algorithms in the RL framework. The advantage of using neural networks enables the RL to search for optimal policies more efficiently in several real-life applicat...
متن کاملReinforcement Learning in Neural Networks: A Survey
In recent years, researches on reinforcement learning (RL) have focused on bridging the gap between adaptive optimal control and bio-inspired learning techniques. Neural network reinforcement learning (NNRL) is among the most popular algorithms in the RL framework. The advantage of using neural networks enables the RL to search for optimal policies more efficiently in several real-life applicat...
متن کاملOn using discretized Cohen-Grossberg node dynamics for model-free actor-critic neural learning in non-Markovian domains
We describe how multi-stage non-Markovian decision problems can be solved using actor-critic reinforcement learning by assuming that a discrete version of CohenGrossberg node dynamics describes the node-activation computations of a neural network (NN). Our NN (i.e., agent) is capable of rendering the process Markovian implicitly and automatically in a totally model-free fashion without learning...
متن کامل